1. Field of the Invention
The present invention relates to digital video recording, and more particularly, to a method of stabilizing digital video recording in two dimensions when recording a scene with an unstabilized digital video camera.
2. Description of the Prior Art
Home video camcorders have become increasingly popular in recent years, with people recording everything from their children's antics to space shuttle launches. One reason for their increasing popularity is that camcorders are becoming smaller. While anyone who has hauled a heavy video camera around appreciates the lower weight, it has come at a cost: the mass of the camera no longer helps to keep the camera, and hence the image, steady. A shaky video makes for poor viewing, since the jittery image is distracting to viewers.
Lately, with the conversion to digital video camcorders, digital image stabilization has become a popular feature. However, the current methods have a number of disadvantages. They only stabilize in one axis, have significant computational overhead requiring bulky circuitry and high power requirements, and the compensation algorithm can become overwhelmed, or “saturated”, meaning that it can fail to compensate properly especially when the camera is being panned across a scene, resulting in an unstabilized image.
A key limitation is that by only stabilizing in a single axis, most current methods of stabilizing digital video do not address a key part of the problem: one of the most distracting parts of an unstabilized video image is unintended motion away from the general motion of the scene. In other words, unintended motion occurring orthogonal to the vector of the intended pan is more distracting than motion occurring inline with the vector.
A typical prior-art stabilization method uses block based motion estimation which divides the captured video frame into a plurality of squares, called macroblocks, compares these macroblocks against a set of prior video frames, determines how much the image has shifted, and selects a subregion of the captured video frame to record as the current frame of the video. The comparison chooses macroblocks that are 16 pixels wide and 16 pixels high, and searches up to 64 pixels in all directions using half-pixel resolution. However, this method is extremely computationally intensive and is unsuitable for implementation in a real-time consumer device due to the complexity, size, and expense of the computer hardware required.
A second prior-art stabilization method uses block based motion estimation which involves choosing a subset of 28 macroblocks, sized 64×64 pixels, and searching within 30 pixels in all directions using full-pixel resolution. Again, however, this method is extremely computationally intensive and is unsuitable for implementation in a real-time consumer device due to the complexity, size, and expense of the computer hardware required; additionally, this method is not directly compatible with the MPEG video standards used in digital video recorders due to the large size of the macroblocks.
Another method, described by Jones in U.S. Pat. No. 6,809,758 attempts to use histograms to stabilize film being converted to digital video, but the method is limited, requiring manual scene-change detection and using an algorithm which is suited to the more specific case of correcting frame jumping during the film conversion process. Jones' algorithm detects rapid jumps in estimated motion and smoothes these out, while not affecting smaller transitions that fall within a threshold. However, small transitions, such as vertical oscillations due to hand shaking or in an extreme case due to a boat rocking on the waves, are a type of motion which it is particularly desirable to eliminate.
An improved method for stabilizing digital video images in two dimensions is clearly necessary in order to improve the viewing experience, especially regarding eliminating the undesirable orthogonal random motion.